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EXECUTIVE  SUMMARY  ot  THE  AUAPT1VE  COMPUTERISED  TRAINING  SYSTEM 


ESI.  T he  Prpbl em  Domain 

The  Army  Research  Institute  sponsored  research  and  develpoment  of  the  "Adaptive 
Comouterized  Training  System"  (ACTS:  contract  no.  MDA9i)3-7H-C-2U39)  to 

investigate  methodologies  for  representing  expert  knowledge  in  an  automated 
system  that  could  provide  Computer  Assisted  Instruction  in  an  electronic 
troubleshooting  environment.  The  targeted  environment  was  a  radio  repair  course 
at  the  Army  Signal  School  -  Ft. Gordon,  Georgia.  A  sizeable  training  problem  had 
evolved  in  the  31  Echo  course  due  to  the  following  conditions: 

(1)  the  basic  aptitude  of  recruits  had  declined  significantly  over  a 
five  year  period. 

(2)  a  sel f-pased  form  of  classroom  instruction  had  been  implemented 
which  increased  the  demands  on  instructors  to  perform  more 
administrative  duties. 

(3)  a  previously  developed  Computerized  Training  System  did  not 
lend  itself  to  interactive  troubleshooting. 

(4)  bottlenecks  occured  when  students  waited  to  be  evaluated  while 
others  finished  training  on  the  available  equipment. 

(5)  classrooms  became  overcrowded  and  instructors  were  barely  able 
to  keep  pace  with  the  numbers  of  incoming  recruits. 

(6)  the  actual  equipment  for  student  training  use  was  in  short  supply 
and  student  access  to  potentially  dangerous  high  voltage 
components  was  not  encouraged. 

These  factors  and  others  resulted  in  a  training  approach  that  aimed  more  at 
training  students  as  adequate  "radio  mechanics"  rather  than  proficient 
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technicians.  The  research  and  development  effort  initiated  by  AR1  was  geared 
towards  providing  a  solution  to  this  problem  through  implementation  of  a  general 
purpose,  knowledge  based,  CAI  system  for  electronic  troubleshooting.  A  summary 
of  the  ACTS  model,  its  principal  components,  the  training  environment  and  areas 
for  theoretical  improvement  is  presented  in  the  following  sections. 


ES2.  Model  of  the  ACT  $  Knowl edge  Based  System. 

As  the  Adaptive  Computerized  Traininq  System  (ACTS)  emerged  from  the  laboratory 
to  find  a  home  in  the  “real-world'  appl ications  area  of  electronic 
troubleshooting,  a  new  approach  was  adopted  to  accomplish  the  transition.  The 
initial  AC  I S  research  focused  or  ways  to  model  expert  and  student  decision 
making  behavior  as  Capturabie  importance  dimensions.  These  dimensions,  or 
attributes,  could  then  be  processed  to  compute  utility  aggregates  for  attributes 
such  as  a.)the  fault  information  to  be  gained  by  a  decision,  b.)  the  value  of 
fault  information  to  be  gained  by  a  decision,  and  c.)  the  value  of  component 
isolation  information  to  be  gained  by  3  decision.  This  mul ti -attribute  utility 
(MAU)  model  (Hopf-Wei chel ,  Purcell,  Freedy,  Luccaccini,  1 y / 9 )  could  represent 
generic  circuit  troubleshooting  strategies  and  was  found  to  be  more  effective 
than  the  expected  utility  (£U)  approach  used  in  the  original  ACTS.  The  MAU 
model  was  ideal  for  expert  modelling  since  adaptive  training  techniques  could  be 
applied  and  experimental  results  evaluated.  The  ACTS  MAU  model  in  the 
laboratory  system  required  the  student  to  analyze  probabilites  and  utility 
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estimates  across  a  set  of  decision  options  during  the  troubleshooting  activity. 
Those  thought  processes  could  then  be  captured  in  the  "student  model"  and 
compared  to  the  embedded  "expert  model"  for  convergence/di vprg^nce  behavior. 
Adaptive  feedback  was  provided  as  the  MAU  model  continuously  tracked  the 
students  decision  performance  during  the  course  of  training. 


However,  when  the  actual  training  environment  was  examined  it  was  determined 
that  a  new  approach  would  be  needed  to  place  an  interactive,  electronic 
troubleshooting  system  into  the  classroom.  Subject  matter  experts  (SMt's)  and 
instructional  specialists  at  the  signal  School  felt  that  the  decision-making 
aspects  of  ACTS  were  far  too  complicated  for  the  average  student  in  the  3 It 
course.  They  had  developed  training  techniques  to  cope  with  changing  recruit 
capability  levels,  changes  in  classroom  loading  and  other  conditions  identified 
above  in  section  ESI.  They  desired  to  have  a  system  that  could  accommodate 
changes  in  instructional  technioues  over  time  and  additionally  be  able  to 
reflect  the  differing  approaches  used  by  individual  instructors.  Hence,  the 
artificial  intelligence  techniques  incorporated  in  the  ACTS  MAU  model  were 
overlaid  by  a  knowledge  based  system  that  permitted  a  more  realistic 
representation  of  electronic  troubleshooting  training  as  it  actually  occurred  in 
the  classroom.  The  MAU  model  was  modified  to  quantify  the  util i t  v  and 
probability  feedback  information  in  a  score  that  students  could  more  easily 
relate  to.  In  addition,  many  automated  management  tools  were  created  to  assist 
instructors  with  courseware  authoring,  student  monitoring,  and  instructional 


ES-3 


development . 
Figure  El. 


An  Overview  of  the  AC l S  knowledge  based  system  is  illustrated  in 


In  designing  the  AC  1 S  for  the  training  environment  at  Ft.  Cordon,  an  exhaustive 
front-end-analysis  was  performed  to  acquire  knowledge  from  the  experts  about 
electronics  devices  (in  this  case,  the  R I  b<d4  radio)  and  their  training 
requirements.  Representation  of  this  information  was  refined  through  repeated 
validation  cycles  before  final  construction  of  the  knowledge  base.  Once 
constructed,  the  knowledqe  base  could  then  be  manipulated  by  SME's  or 
instructors  to  specify  their  individual  approaches  towards  presentinq  electronic 
troubleshooting  instruction.  They  could  tailor  the  content  specific  data,  i.e. 
the  graphic  display  representation  of  circuits,  symptoms  associated  with 
different  problems,  the  forward  chaining  representation  in  the  scheme 
composition,  and  t fie  dynamic  sequencing  of  troubleshooting  problems  presented  to 
students.  The  flexibility  of  the  new  ACTS  knowledge  based  system  is 
unparalleled  for  electronic  troubleshooting  by  other  CAI  systems  such  as  FLA TO, 
PLANIT,  or  PILOT. 


ESd.  Components  of  the  ACTS  T rai ni nq  Envi ronment. 

The  development  of  ACTS  for  the  Signal  School  integrated  substantial  instructor 
components  with  student  training  components  to  provide  a  user-friendly  training 


environment.  A  major  requirement  of  the  project  was  to  provide  course  authoring 
programs  that  woo'd  make  courseware  construction  easy  to  accomplish  for  the 
SMfc ’ s  and  instructors  at  the  Signal  School.  The  overall  design  of  ACTS  was 
geared  towards  development  of  a  generic  system  that  could  be  used  to  train 
electronic  troubleshooting  for  many  different  types  of  devices.  Figure  E? 
highlites  the  primary  components  of  ACTS  showing  the  different  program  modules 
for  instructors  and  students.  An  important  aspect  of  the  ACTS  is  its  capability 
to  provide  students  with  traditional  computer  aided  instructional  training, 
complete  with  testing  and  adaptive  remedial  training.  This  feature  permits  the 
development  of  a  complete  course  of  training  material  to  be  delivered  to  the 
student  prior  to  the  electronic  troubleshooting  component.  Uur  tocos  at  the 
signal  school  was  to  provide  courseware  for  the  transmitter  section  of  the  R 1  4 
radio,  i .e.  instructional  content  material  normally  covered  in  one  objective  of 
the  ii  Echo  course,  ACTS  could  easily  provide  instruction  for  all  objectives  in 
the  il  Echo  course  through  further  courseware  development. 
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FIGURE  E2  -  COMPONENTS  OF  ACTS 


ES4.  Acceptance  of  ACTS  i n  the  T  rai ni ng  Env  i ronment 

In  January  of  1V»1,  during  the  front-end-analysis  phase  of  ACTS  transfer  to  Ft. 
Gordon,  the  laboratory  version  of  ACTS  was  installed  and  demonstrated  at  Ft. 
Gordon.  The  training  managers,  instructors,  and  SMt ' s  who  participated  in  those 
demonstrations  provided  feedback  that  helped  define  the  requirements  for  the 
knowledge  based  version  of  ACTS.  Their  responses  included  the  following: 

(1)  the  MACE  model  approach  with  its  decision  making  activity  was  too 
complicated  for  31E  students 

(2)  a  need  was  identified  for  actual  test  point  measurement  values 
to  be  provided  during  the  troubleshooting  process 

[5)  a  heirarcnical  scheme  should  be  implementedf  that  represents  actual 
troubleshooting  processes  of  "secti onal i zat ion" ,  "localization", 
and  "isolation". 

(4)  a  set  of  course  autnoring  prigrams  were  required  that  would  permit 
instructors  to  design  troubleshooting  problems. 

(b)  instructor  management  aids  would  be  required  to  perform  student 
mon i tori ng 

(b)  the  courseware  developed  for  the  3  IF.  cooursr  should  be  validated  by 
SME's  prior  to  use  in  the  classroom. 
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These  six  objectives  guided  the  design  and  implementation  of  the  knowlede  based 
version  of  ACTS  that  was  demonstrated  at  Ft.  Gordon  in  April  of  19B1.  fhe  new 
system  met  with  overwhelming  success  —  described  by  one  SME  as  "...yes,  it  can 
teach".  By  December  of  lybl  the  full  ACiS  had  been  installed  at  Ft.  Gordon  on 
their  computer,  complete  with  all  course  authoring  programs,  validated 
courseware,  and  operational  with  multiple  student  terminals.  Several  training 
sessions  were  held  to  Instruct  SME's  in  the  techniques  of  developing  new 
courseware  and  several  students  used  ACI S  on  a  trial  basis  to  validate  its 
operation  prior  to  full  classroom  evaluation.  One  student,  who  had  completed 
traditional  classroom  training  on  the  RtS2b  transmitter,  worked  all  of  the 
troubleshooting  problems  on  ACIS  without  any  prior  terminal  tutorial  training  or 
use  of  the  student  manual. 

The  student  evaluation  plan  had  been  scheduled  when  a  series  of  events  disrupted 
the  continuation  of  the  project.  These  included: 

(1)  schedule  changes  for  the  31  Echo  course  in  the  Signal  School 

(2)  The  31  Echo  course  was  moved  to  a  new  location  at  Ft.  Gordon 

(3)  continuation  funding  for  the  evaluation  plan  was  delayed. 

(A)  loss  of  computer  resources  at  Ft. Gordon  to  support  ACIS  in  a  dally 
training  environment. 
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1.  INTRODUCTION 


1.1  Overview. 

This  report  describes  the  work  preformed  during  the  third  year  of  a  three-year 
program  of  research  and  development  directed  at  the  evaluation  of  the  Adaptive 
Computerized  Training  System  (ACTS). 

The  ACTS  is  a  training  system  used  to  teach  electronic  troubleshooting 
procedures  with  particular  emphasis  focused  on  the  decision-making  aspects  of 
these  procedures.  Artificial  intelligence  techniques  are  applied  to  the 
training  situation  by  simulating  tne  electronic  equipment  under  repair  and  by 
adaptive  computer  modeling  of  student  behavior.  The  primary  areas  of  concern 
during  this  three-year  program  were  the  evaluation  of  ACTS  both  in  the 
laboratory  and  in  the  field,  and  the  continued  development  and  modification  of 
the  original  ACTS  software.  Third  year  activities  were  to  focus  primarily  on 
field  evaluation  and  on  the  application  of  ACTS  to  the  field  radio  repairer 
(31E)  course  at  the  U.S.  Army  Signal  Center  and  Ft.  Gordon  (USASCSFG). 


1.2  Objectives  of  Year  Three. 

There  were  three  major  objectives  in  year  3.  The  first  was  to  conduct  a 
thorough  analysis  of  instructional  methods  and  materials  used  in  the  31E  course 
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at  Ft.  Gordon.  This  included  collaboration  with  RT524  Subject  Matter  Cxperts  to 
complete  the  instructional  design  for  the  courseware  used  in  ACTS.  The  second 
objective  was  to  implement  ACTS  software  (version  1.0)  for  student  and  research 
operation.  The  third  was  to  conduct  a  field  evaluation  of  the  ACTS  in  the  311- 
course  at  Ft.  Gordon,  revise  MTS  based  on  the  result,  and  conduct  comprehensive 
training  of  Data  Systems  Oivision  (050)  personnel  and  31E  instructors  in  the 
continued  use  and  development  of  ACTS.  Activities  and  accomplishments  in 
support  of  these  three  objectives  are  described  in  detail  in  chapters  2  and  3  of 
the  present  report. 


1.3  Approach. 

Computer  assisted  instruction  (CA1)  has  developed  into  an  integral  part  of 
current  training  technology.  CM  uses  new  concepts  in  the  computer  sciences 
along  with  artificial  intelligence  techniques  to  investigate  the  cognitive 
aspects  of  problem-solving  and  decision-making  behavior.  A  forerunner  in  CAI 
technology  is  the  Adaptive  Computerized  Training  System,  referred  to  as  ACTS. 
In  this  system,  individualized  training  is  achieved  utilizing  adaptive 
techniques  in  a  simulated  electronic  maintenance  environment. 

ACTS,  as  originally  designed,  is  a  system  that  tracks  the  behavior  of  a  student 
and  shapes  it  to  resemble  that  of  an  expert.  A  major  portion  of  the  first 
year's  effort  consisted  of  developing  an  extended  ACTS  system  to  Implement 
“multi-attribute  utility"  models  to  represent  student  and  expert  behavior  in 
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place  of  the  original  “expected  value"  models  in  ACTS.  An  additional  effort 
during  the  first  year  of  this  project  was  to  expand  the  system  to  accommodate 
multiple  tasks  and  multiple  students  by  developing  multiple  circuit  capability, 
improving  the  ACTS'  software  organization  and  by  generating  instructional 
materials  for  two  circuit  modules  (IR-i'b  and  A-yUDO).  That  initial  system  was 
successfully  implemented  and  tested  in  a  laboratory  environment. 

During  the  second  year  the  ACTS  concept  was  redesigned,  emphasizing  its 
conversion  to  a  higher  order  language  for  field  implementation  at  Fort  Gordon, 
Georgia.  A  user  manual  was  developed,  and  the  initial  transfer  of  software  and 
courseware  to  the  Fort  Gordon  Army  School  began.  An  extensive  series  of 
laboratory  studies  focused  on  the  sequencing  of  problem  presentation,  transfer 
of  training,  and  optimal  use  of  both  ACTS  traininq  and  training  on  simulated 
equipment,  the  effects  of  extended  training,  and  the  use  of  ACTS  for  training 
troubleshooting  on  an  additional  circuit  model. 

The  third  year's  activities,  as  described  in  this  report,  concluded  the 
implementation  and  installation  of  ACTS  software  at  the  Data  Systems  Division 
(050)  at  Ft.  Gordon.  Collaboration  with  the  3 IE  Subject  Matter  Experts  and  OSD 
personnel  resulted  in  further  development,  modification,  and  refinement  of  both 
the  ACTS  software  and  courseware  to  meet  the  requirements  defined  by  3 IE  and 
OSD  personnel.  This  effort  resulted  in  the  full  development  and  validation  nf 
Version  1.0  of  both  software  and  courseware  which  presents  the  hierarchical 
fashion  for  the  RT5'<;A/VRC  Radio  Transmitter  in  an  Evaluations  by  instructional 
personnel  resulted  in  further  software  modifications  to  reduce  the  amount  of 
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decision  making  information  to  be  presented  in  a  classroom  environment. 


1.4  Accomplishments. 

The  Adaptive  Computerized  Training  System  (ACTS)  has  been  under  development  for 
a  number  of  years.  Initiated  in  1974  under  the  sponsorship  of  the  Army  Research 
Institute,  the  project's  major  objective  has  been  to  develop  a  training  syst»»m 
that  would  address  general  troubleshooting  skills,  as  opposed  to  repair 
procedures  for  a  specific  piece  of  equipment.  These  skills  would  transfer  to 
various  equipment  types  and  would  reduce  the  substantial  equipment  support 
necessary  for  highly  directive  maintenance  skill  development.  Although  this 
goal  has  remained  constant,  ACTS  has  evolved  through  various  implementations 
with  continued  focus  research  and  development  activity.  The  basic  structure  of 
ACTS,  however,  has  not  chanqed.  Comprising  the  core  of  ACTS  are  four  major 
components:  the  task  model,  the  expert  model,  the  student  model,  and  the 

instructional  model.  By  exercising  these  models,  ACTS  executes  an  adaptive 
instructional  strategy  that  captures  the  student’s  decision  making  value 
structure,  compares  this  structure  to  that  of  an  expert,  and  then  varies  the 
instructional  sequence  to  eliminate  divergence.  The  culmination  of 
developmental  research  over  the  period  from  19/4  to  the  present  has  resulted  in 
two  distinct  versions  of  ACTS.  The  research  model  of  ACTS  reflects  the  original 
adaptive  instructional  strategy.  However,  many  of  the  previous  design  features 
of  ACTS  have  been  improved.  These  Include 
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1.  Multiple  student  option.  Two  or  more 
students  can  use  the  system  simultaneously. 

2.  Multiple  circuit  option.  Students 
using  ACTS  may  run  simultaneously  and 
independently  using  as  many  different  circuits 
as  have  been  developed. 

3.  Variable  mode  problem  presentation.  The 
sequence  In  which  problems  are  presented 

to  the  student  can  be  of  random  difficulty, 
graduated  difficulty  or  specified  by  the  instructor. 

4.  Hierarchical  circuit  models  are  used  to 
present  each  problem  as  a  sequencing  process 
of  secti onal i zation ,  localization,  and 
isolation. 


Noteworthly  areas  of  accomplishment  during  the  most  recent  year  of  effort 
include  full  installation  of  ACTS  in  the  31E  traininq  environment,  desiqn  and 
implementation  of  extensive  course  authoring  tools  to  provide  Ft.  Gordon 
instructional  personnel  with  the  capability  to  modify  and  create  circuit  model 
courseware,  and  training  of  Ft.  Gordon  OSD  and  31E  personnel  in  the  operation  of 


all  ACTS  components  including  course  authoring. 


1.5  Recommendations  to  Improve  ACTS  Performance. 

Recommendation's  for  a  fourth  year’s  effort  include  the  development  and 
installation  of  upgraded  versions  of  software  and  courseware.  In  addition,  a 
Baseline  Expert  Model  should  be  developed  from  Subject  Matter  Experts  who  use 
ACTS  to  troubleshoot  all  courseware  problems.  This  would  permit  an  extended 
evaluation  of  ACTS  to  be  conducted  based  on  the  new  courseware,  software,  and  an 
Expert  Model.  This  student  evaluation  could  then  be  compared  against  the 
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traditional  classroom  training  to  analyze  training  and  cost  effectiveness  of 
ACTS.  The  primary  benefit  of  a  Baseline  Expert  Model  comoonent  would  be  that 
personalized  courseware  could  be  developed  by  each  Expert.  Hence,  instrucotrs 
and  SME's  could  tailor  classroom  training  to  reflect  their  differing 


instructional  methods. 


2.  IMPLhMENTAT ION  OF  ACTS  FOR  FI.  GORDON 


2.1  Overview 

This  chapter  describes  two  aspects  of  the  implementation  of  ACTS  for  Ft. 
Gordon.  First  was  the  thorough  analysis  of  Instructional  methods  and  materials 
used  in  the  3 IE  course.  This  included  as  gaining  a  complete  understandi ng  of 
the  specifications  desired  by  3 IE  and  OSD  personnel.  The  second  was  the  full 
implementation  and  evaluation  of  ACTS  that  resulted  from  end-user 
sped  f  icati  ons. 

2.2  Front  End  Analysis. 

2.2.1  End-User  Requi rements.  Initial  meeting  with  OSD  and  31E  personnel  in 
early  1981  identi  fi eo  several  problem  areas  that  required  addressing  before  full 
ACTS  implementation  and  evaluation.  These  areas  included: 

Screen  Presentation  -  This  involved  the  manner  in 
which  the  equipment  models  were  presented  as  well  as 
the  way  students  interacted  with  the  syStEM. 

Model  Capabilities  -  This  involved  the  flexibility 
of  the  f e j1 t  matrices,  the  type  of  measurements 
present,  and  the  complexity  of  the  decision-making 
information  presented  to  the  students 

Courseware  Authoring  -  Instructional  personnel  at  the 
signal  school  indicated  the  importance  of  providing 
easy-to-use  course  authoring  tools  so  that  SME's  could 
develop  extensive  training  materials  for  many  applications. 


Based  on  new  requi rments  defined  by  DSD  and  3 IE  personnel  ,  the  following  design 
changes  were  made  to  AC  I S  to  provide  a  student  version  of  the  troubleshooting 
system: 


1.  The  block  module  presentation  was  modified  to 
facilitate  component  isolation  down  to  the 
schematic  level. 
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2. 


Screens  2  &  3  of  the  feedback  displays  were 
eliminated  as  was  the  expert,  choice  feedback. 

Only  the  score  information  was  retained. 

It  was  indicated  by  USU  S  3 1 £  personnel  that 
the  decision  making  informatkr  was  too  complex 
and  difficult  for  31E  students  to  fully  understand. 


In  addition,  ARl  requested  that  the  more  complex  version  (renamed  as  the 
Research  Version)  also  be  modified,  installed,  and  evaluated  at  Ft.  Gordon. 


2.2.2  System  Capabi 1 ities.  The  ACTS  troubleshooting  system  as  implemented  at 
Ft.  Gordon,  has  the  following  capabilities: 


1.  tase  of  Student  Use.  ACTS  accessibility 
is  established  by  the  instructor  providing 
students  with  direct  entry  into  the  trouble¬ 
shooting  material  after  logging  into  the 
system. 

2.  Multi-Student-Use.  The  number  of  simultaneous 
users  of  ACTS  is  a  function  of  the  number 

of  graphics  display  terminals  and  not  a 
system  imposed  constraint.  Students  in¬ 
volved  in  the  troubleshooting  process  can 
be  accessing  different  circuits  and  different 
problems  independently. 

3.  Student  Monitoring.  The  instructor  can  monitor  the 
progress  of  students  while  they  are  trouble¬ 
shooting.  The  instructor  can  also  invoke 

a  number  of  control  conditions  on  a  given  student’s 
troubleshooting  session  and  obtain  immediate  data 
regarding  student  performance. 

4.  Circuit  Development  Aids.  Separate  utilities  and 
tutorial  programs  are  provided  to  assist  in  the 
development  of  new  circuit  modules  in  ACTS.  Course 
Authoring  capabilities  of  the  CA I  system  will 
continue  to  be  enhanced  and  extended. 
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5.  T ransportabi  1  i ty  -  All  ACTS  program  software  is 
written  in  FORTRAN  guided  by  the  Proqram  Design 
Language  specifications.  Ihe  system  is  intended 
to  be  highly  modular  and  portable,  but  is  -graphics 
terminal ,  dependent  rather  than  system  dependent. 


2.2 .3  Target  System  Configuration.  The  U.S.  A~my  Signal  School  at  Ft.  Gordon, 
Georgia  originally  utilized  a  network  of  6  PDP  11/3‘j  mini -computers  in  their 
Computerized  Training  System  (CIS)  environment.  As  shown  in  Figure  2-1,  one  of 
the  POM  11/35 ' s  can  also  be  used  as  a  satellite  computing  system.  The  Signal 
School  provided  computing  resources  on  this  PDH  1 1 /3b  to  support  the  ACTS 
troubleshooting  system.  The  Configuration  arrangement  of  the  1 1 /3b  is  as 
f  0 1 1  ows : 


(1)  12  UK  words  of  main  memory. 

(2)  Memory  management  hardware. 

(3)  2  RKU5  2.5  million  byte  disk  drives. 

(4)  lb  line  asynchronous  serial  1/U  multiplexor  (DH  11). 

(5)  RSX11M  operating  system  that  supports  DtC's  FORTRAN 
IV. 

(6)  Dialup  hardware  for  remote  terminal  access. 


-The  MUM  1 1 /3b  central  processor  was  not  configured  with  either  floating  point 
processing  hardware  (FPM)  or  the  extended  instruction  set  (EIS)  which  implements 
integer  multiply  and  divide  instructions  in  hardware.  However,  FPM  and  EIS 
instructions  are  emulated  in  system  software  providing  complete,  although 
marginally  slower,  processing  power.  An  analysis  determined  that  FPP  and  EIS 
hardware  are  not  required  for  operation  of  the  AC1S  troubleshooting  sytem  since 
access  will  be  over  slow  speed  communication  lines  from  the  classroom. 


Large  Capacity 
Disk  Storage 


Local 

Diak 


To  Display 
Terminals 


FIGURE  2.1  -  CURRENT  FT.  GORDON  CONFIGURATION 
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it  is  important  to  note  that  the  FURTRAN  compiler  existinq  on  RSXll  system  is 
DEC'S  standard  ANSII  FORTRAN  IV  and  not  FORTRAN  [V  PLUS  (F4P).  F4P  features 
were  not  utilized  during  (ACTS)  implementation  resulting  in  minimal  software 
modifications  when  A<  IS  was  transferred  to  Fort  Gordon.  However,  in  order  to 
support  the  generalized  CA1  software  system  of  ACTS  it  was  necessary  to  upgrade 
FUKTRAN  IV  to  version  2.S.  This  was  completed  on  the  Ft.  Cordon  RSX  system. 

2.3  Overall  Design. 

2.3.1  Field  System.  The  loqical  configuration  of  the  ACTS  training  environment 
shown  in  Figure  2-2  contains  three  primary  components:  (1)  Student  Tra’ning, 
(2)  Instructor  Supervisory,  and  (3)  System  Development  and  Maintenance.  Each 
component  interacts  with  files  in  the  system  data  base.  Components  include  not 
only  programs  that  present  the  instructional  material  but  also  programs  to 
update  and  query  performance  files,  system  source  files  and  maintenance  files. 
Features  of  the  three  logical  components  are  given  in  the  following  paragraphs. 

Student  Training  Component.  Included  here  are  thp  instructional  training  units 
comprising  the  course  materials  for  the  students.  The  system  automatically 
cycles  the  student  through  several  phases  based  upon  assignments.  The 
operational  segments  are: 


Preliminary  Instructional  Objectives 


TRAINING  ENVIRONMENT 


FIGURE  2.2  -  ACTS  SYSTEM  ORGANIZATION 


The  ACTS  courseware  disk  is  used  for  further  development  of  the  instructioal 
courseware  and  development  of  new  or  experimental  courseware.  It  contains  tne 
following  directories: 

1.  Experimental  Course  Authoring  -  contains  all  the 
AC i S  course  authoring  tools  and  associated 
courseware  directory  for  new  or  experimental 
development. 

2.  Instructional  Course  Authoring  -  Contains  all  of  the  ACTS 
course  authoring  tools  and  necessary  courseware 

directories  for  further  development  of  courseware  curently  being  used. 


id. 4.4  Instructor  System  Access  -  Instructors  have  general  capabilities 
regarding  the  control  and  monitoring  of  student  usage.  Programs  available  to 
the  instructor  include  the  following: 


1.  Add  student 

2.  Remove  student 

3.  Review  c I  ass 

4.  Review  student 

5.  Modify  student  assignment 

2.4,  s  Student  System  Access  -  Students  who  have  brer,  given  access  to  AC  I S  can 
only  access  the  troubleshooting  system. 


2.b  Design  Considerations  for  ACIS  Enhancement 

Based  on  requirements  defined  by  DSD  and  31E  personnel,  the  following 
enhancements  are  recommended  to  Improve  ACIS  performance  and  make  It  more 
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-  Graphical  Troubleshooting  Model 
Performance  data  acquisition 

1 nstructor  Supervi sory  Component.  These  programs  provide  the  instructor  with 
direct  control  over  the  training  environment.  Functional  capability  of  these 
program  include: 

-  Student  access  and  troubleshooting  specification 

-  Student  session  monitoring 

-  Student  performance  data  retrieval 

System  Development  and  Maintenance  Component.  Overall  system  functionality  and 
completeness  are  established  with  the  addition  of  these  operational  procedures: 

-  Circuit  module  development  aids 
Instructional  unit  installation  aids 

-  System  model ing 

Z.i.Z  Research  System.  The  logical  configuration  of  the  ACTS  research  system 
is  very  similar  to  that  described  for  the  field  system  while  providing  some 
features  not  found  in  the  field  system.  These  features  include; 

-  Instructional  test  units  for  student  training 

-  "Three-screen”  ACTS 

This  system  is  installed  on  AKPA's  DDF  time-sharing  POP  11/70.  Primary 
utilization  of  the  research  system  has  been  the  development  and  preliminary 
testing  of  ACTS  modifications  before  transfer  to  Ft.  Gordon. 
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2.4  Information  Flow  and  Management. 


2.4.1  Student  Performance  Recording.  Each  student  has  a  performance  file 
constructed  from  an  initial  master  record  when  student  access  is  established  by 
the  instructor.  Upon  completion  of  each  training  objective,  the  current 
performance  record  for  that  student  is  stored  on  his  private  performance  file. 
As  a  student  begins  an  interactive  training  session  his  current  performance  file 
is  read  into  the  training  model,  and  the  ACTS  software  invokes  the  correct 
objective  or  troubleshooting  problem. 

Each  students  file  contains  all  control  information  necessary  to  specify  the 
students  current  instructional  status,  the  state  of  the  current  circuit,  and 
generally  all  student  and  environmental  control  Infromation  needed  to  display  or 
print  performance  information  when  accessed  by  the  instructor. 

2.4.2  System  Operational  Control .  Flow  of  control  information  in  the  ACTS 
environment  1?  represented  in  Figure  2.3.  Various  student  and  instructor 
activated  programs  update  and  process  the  three  main  training  files.  In  a 
typical  scenario  the  following  sequence  of  action  might  occur. 

1.  Instructor  establishes  a  class  of  students  to  receive 
troubleshooting  training.  An  activation  proqram 
queries  the  instructor  for  student  names  and  account 
numbers,  then  allows  the  Instructor  to  update  the 
Student  Control  File. 

2.  Students  log  into  the  computer  system  and  activate 
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FIGURE  2.3  -  LOGICAL  INFORMATION  FLOW 


the  troubleshooting  program.  The  program  loads 
student's  instructional  and  unit  control  information 
from  the  Student  Control  File  and  proceeds  into  inter¬ 
active  troubleshooting  model. 

3.  Instructor  activates  performance  access  program 
to  obtain  student  training  results.  Student 
Performance  File  processed  to  display  or  print 
Information,  on  one  or  more  students,  requested 
by  instructor. 


2.4.3  File  System  Description  -  This  section  provides  a  general  overview  of 
the  file  organization  of  ACTS  on  the  PUP  ll/ib  under  RSX. 


The  AC  I S  system  resides  on  four  different  RKUS  disks.  Two  of  these  disks 
contain  Ac  IS  source  code  and  course  authoring  source  code  as  well  as  the 
necessary  control  files  to  compile,  task  build,  and  test  proqrams.  The 
remaining  two  disks  compose  the  operational  part  of  the  ACIS  system,  ihe  AUS 
class  disk  is  used  for  troubleshooting  in  the  classroom  and  instructor 
monitoring.  It  contains  the  following  directories  or  accounts: 


1.  Instructor  Directory  -  Provides  the  instructor  with 
access  tc  student  monitoring  programs  and  to 

ACTS  troubleshooting  programs. 

2.  Student  Directories  -  There  are  20  dif¬ 
ferent  directories  that  allow  access  to 
ACTS  troubleshooting  and  contain  control 
and  performance  files. 

3.  ACTS  directory  -  contains  the  ACIS  executable 
program. 

4.  Courseware  Directory  -  Contains  the 
Instructional  Courseware  for  the  Rlb24  Radio 
Transmitter. 


2-11 


useable  In  the  31E  environment: 


1.  Upgrade  and  expand  the  courseware  for  the  R 1 4 
radio. 

2.  Upgrade  AClS  software  to  improve  multi¬ 
student  use. 

3.  Consolidate  and  modify  both  the  instructor 
monitoring  programs  and  the  course  authoring 
tools  to  improve  efficiency  and  ease  of 
use. 

4.  Reduce  the  decisien-making  Information 
presented  to  the  student,  which  will 
suppress  the  background  expert  models. 

5.  Develop  a  baseline  expert  model  from  analysis 
of  Sub/ject  Matter  Expert  decision  making 
activity  while  utilizing  ACIS  troubleshooting 

6.  Conduct  an  extended  evaluation  of  ACTS  based  on 
new  courseware,  software,  and  an  expert  model. 


3.  THE  TRAINING  ASPECTS  OF  ACTS 


3.1  Overview 

The  Adaptive  Computerized  Training  System  (ACTS)  focuses  on  Improving  and 
sharpening  higher-order  cognitive  skills  in  electronic  troubleshooting. 
Although  maintenance  tasks  rely  heavily  on  a  technician's  knowledge  and  training 
regarding  the  electronic  systems,  such  tasks  can  be  viewed  primarily  as 
decision  tasks.  If  the  technician  possesses  sufficient  knowledge  of  system 
parts  and  functions,  he  applies  it  by  making  a  series  of  decisions  about  which 
symptoms  to  look  for,  whether  to  repair  or  replace  a  malfunctioning  part,  and 
what  sort  of  test  measurements  need  to  be  made.  The  student's  task  is  to 
troubleshoot  a  complex  circuit  by  making  various  measurements,  replacing  the 
malfunctioning  part,  and  making  final  verification  measurements. 
Troubleshooting  provides  excellent  application  for  the  ACTS  methodology  because 
it  Is  heavily  dependent  on  judgement  and  probabilistic  inference.  In  addition, 
troubleshooting  is  of  great  practical  importance  in  numerous  commercial  and 
military  systems,  and  it  lends  Itself  to  economical  implementation  for  training 
purposes 

Work  to  date  has  produced  an  optional  system  which  demonstrates  the  feasibility 
of  applying  artificial  intelligence  techniques  to  computer-assisted  instruction 
in  a  minicomputer-based  training  system. 
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Field  evaluations  by  DSO  and  31E  instructional  personnel  indicated  the  decision 
models  and  feedback  components  of  ACTS  presented  information  that  was  too 
complex  for  feasible  assimilation  by  the  typical  31E  student.  This  has  resulted 
in  a  severe  reduction  in  the  amount  of  feedback  information  provided, 
essentially  suppressing  the  background  decision  models. 

The  training  given  in  the  circuit  fault  diagnosis  and  repair  task  is  based  on 
the  assumption  that  the  student  has  a  good  basic  backqroud  in  electronics  but 
that  his  experience  with  troubleshooting  is  limited.  This  skill  level  can  be 
assessed  either  in  terms  of  previous  training  received  or  in  terms  of 
performance  on  an  entering  test  of  electronic  and  troubleshooting  knowledge.  By 
testing  the  student  on  the  laws  of  electrical  theory,  circuit  component 
behavior,  circuit  sub-systems,  circuit  diagrams,  use  of  test  equipment,  and  the 
like,  we  can  determine  the  extent  of  knowledge  that  the  student  already  has  and 
then  adapt  the  level  of  training  to  challenge  him  or  her. 


3.2  ACTS  System  Description 

3.2.1  Task  Simulator.  In  the  ACTS,  the  student's  decision  task  involves 
troubleshooting  an  electronic  device.  The  troubleshooting  task  centers  on  a 
model  of  an  electronic  circuit  in  which  faults  can  be  simulated.  Currently,  the 
circuits  modeled  Include  all  the  components  of  the  RTb24  Radio  Transmitter.  The 
operation  of  the  RTb24  Radio  Transmitter  Is  simulated  by  the  computer  program, 
using  a  table-driven  simulation  of  the  fault  system.  The  program  simulates  the 
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results  of  checking  symptoms,  taking  measurements,  and  replacing  modules. 

Training  in  the  present  system  occurs  with  certain  restrictions  on  the  extent  of 
circuit  simulation.  The  student  interacts  with  a  terminal  which  contains  a 
display  of  the  simulated  circuit,  thus  he/she  cannot  make  such  troubleshooting 
observations  as  smelling  faulty  capactors,  looking  for  burned  resistors,  or 
touching  overheated  semiconductors. 

The  circuit  simulation  was  designed  to  meet  several  objectives.  In  additon  to 
providing  an  environment  for  observing  troubleshooting  behavior,  the  simulator 
gives  the  results  of  the  student's  choice  of  alternatives  by  displaying  the 
results  of  measurements.  Finally,  the  circuit  model  is  designed  to  simulate  the 
essential  characteristics  of  decision-making  under  uncertainty.  Thus,  the 
outcomes  of  the  measurements  are  probabilistic,  reflecting  the  fact  that,  in 
practice,  fault  locations  are  uncertain  for  the  troubleshooter. 

3.3  Instructional  Approach 

Training  on  the  ACTS  is  provided  through  a  system  of  troubleshooting  units.  Each 
troubleshooting  problem  consists  of  a  hierarchical  series  of  components  leading 
from  the  top  level  (sectional i zation)  down  through  sub-assemhl i es  (localization) 
to  the  schematic  level  (Isolation.)  The  student  can  choose  to  take  a  voltage  or 
resistance  measurement,  or  replace  any  circuit  module.  Following  a  student's 
command  to  perform  these  activities,  the  ACTS  program  displays  the  results  of 
the  simulated  activity.  Feedback  for  test  point  measurement  consists  of  the 
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numeric  value  for  that  measurement.  Feedback  for  module  replacement  consists  of 
indicating  whether  the  replaced  module  was  good  or  faulty.  Replacement  of  the 

faulty  module  causes  transition  to  the  next  level.  At  the  lowest  level  for  any 

* 

given  problem,  isolation  of  the  faculty  module  or  component  results  in  a 
congratulatory  messge  and  a  final  score. 


3.4  Evaluation  Plan 

A  summary  of  the  extended  ACTS  evaluation  model  is  presented  below: 

Forty  students  will  be  designated  for  the  evaluation.  All  students  will 
receive  the  basic  electronics  training  course  and  will  complete  the 
training  objectives  as  they  normally  would  in  the  TIE  curriculum.  Students 
will  then  enter  objective  4  and  will  receive  circuit  familiarization  and 
the  initial  troubleshooting  exercise.  This  is  indicated  by  Steps  1  through 
4  in  Figure  3-1. 

Co  supplement  the  student  data  already  available,  after  these  exercises  are 
computed,  the  student  will  fill  out  a  background  questionnaire.  A  pretest 
will  then  be  given  on  troubleshooting  problems  (step  5  on  Figure  3-1). 
This  pretest  will  be  written  and  will  consist  of  10  multiple  choice 
questions  randomly  chosen  from  a  set  of  20  questions.  These  questions  will 
be  regarding  the  transmitter  only.  The  test  will  take  approximately  1/2 
hour. 
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Basic  Electronics 
Training 

Training  Objectives 
1-3 


Circuit  Familiarization 


Initial  Troubleshooting 
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Problems  Written 


7  Post-Test  1 


Traditional 

Troubleshooting 

Training 


Post-Test  2 


FIGURE  3.1  -  ACTS-EVALUATION  MODEL 
3-5 


After  the  pretest,  the  students  will  be  randomly  assigned  to  one  of  2 
groups,  twenty  students  will  receive  the  traditional  troubleshooting 

training  for  a  period  of  6  hours.  The  other  twenty  students  will  receive 

AC  1 S  training.  They  will  work  with  the  ACTS  for  a  six-hour  period, 
attempting  to  complete  as  many  problems  as  possible  durinq  that  time  frame. 
The  six-hour  period  for  all  students  will  focus  on  transmitter  problems 

only.  This  step  is  indicated  in  the  diagram  as  Step  6.  At  the  completion 
of  the  6  hours  of  training,  all  students  will  receive  the  first  post-test. 
This  will  consist  of  1U  problems  randomly  generated  from  the  20  problems 
used  in  the  pretest.  This  will  be  a  multiple  choice  written  test.  It  is 
indicated  as  Step  7  in  the  flow  diagram  and  will  take  approximately  1/2 

hour. 

The  students  who  received  the  ACTS  training  will  then  receive  the 
traditional  training.  Those  students  who  initially  received  the 
traditional*  trainig  will  continue  with  it.  It  is  hypothesized  that  no 
additonal  time  will  be  required  in  traditional  training  for  the  students 
who  originally  received  the  ACTS,  perhaps  even  less  time.  This  will  enable 
all  of  the  students  to  complete  the  4th  objective  in  the  approximate  25 
hours  allotted  to  maintain  the  program  of  training  with  only  four  to  six 
additional  hours  added  to  incorporate  the  pre-  and  post-tests. 

At  the  completion  of  the  traditional  training  program.  The  students  will 


3-6 


receive  the  second  post-test  consisting  of  five  oroblems.  This  test  will 
require  actual  manipulation  of  the  equipment.  The  students  will  be 
expected  to  take  the  appropriate  number  of  measurements  and  replace  the 
correct  module  in  the  minimum  amount  of  time  required  to  repair  the 
equipment.  The  instructor  will  record  the  number  of  measurements  taken, 
modules  replaced,  and  the  amount  of  time  required  to  fix  each  of  the  5 
faults.  This  post-test  will  take  approximately  two  hours.  All  groups  will 
then  go  on  to  Step  10  which  indicates  traditional  evaluation  of  objective  4 
and  the  completion  of  training  objectives  5  through  lb. 

At  the  end  of  the  course,  overall  performance  of  the  three  qroups  will  be 
evaluated  by  comparison  of  each  students  progression  index.  Complete 
instructions,  record  keeping  materials  and  test  materials  will  be  given  to 
the  instructors  to  facilitate  the  evaluation. 
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APPENDIX  A 

Titles  of  Associated  ACTS  Reports 


APPENDIX  1 

The  following  documents,  produced  by  Perceptronics,  are  associated  with  ACTS: 

1.  PIM-1U76-82-01  -  Instructor's  Manual 

2.  PPRM-1U76-82-Q1  -  Programmer's  Reference  Manual 

3.  PSC0B-1U/6-82-01  -  Summary  of  Current  Acts  Courseware  Database 


The  following  documents,  produced  by  Perceptronics ,  are  associated  with 
ACTS: 


1.  PIM- 1076-82-01  -  ACTS  Adaptive  Computerized  Training  System, 
Instructor' s  Manual . 

2.  PPRM- 1076-82-01  -ACTS  Adaptive  Computerized  Training  System, 
Prograrmier' s  Reference  Manual. 

3.  PSCOB- 1076-82-01  -  ACTS  Adaptive  Computerized  Training  System, 
Sunmary  of  Current  ACTS  Courseware  Database 
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